User interface for three-dimensional measurement device

ABSTRACT

A system and method for providing feedback on a quality of a 3D scan is provided. The system includes a coordinate scanner configured to optically measure and determine a plurality of three-dimensional coordinates to a plurality of locations on at least one surface in the environment, the coordinate scanner being configured to move through the environment while acquiring the plurality of three-dimensional coordinates. A display having a graphical user interface. One or more processors are provided that are configured to determine a quality attribute of a process of measuring the plurality of three-dimensional coordinates based at least in part on the movement of the coordinate scanner in the environment and display a graphical quality indicator on the graphical user interface based at least in part on the quality attribute, the quality indicator is a graphical element having at least one movable element.

BACKGROUND

This application is a continuation application of U.S. patentapplication Ser. No. 17/340,917 filed on Jun. 7, 2021, which claims thebenefit of U.S. Provisional Application Ser. No. 63/044,672 filed Jun.26, 2020, the contents of both of which are incorporated herein byreference.

BACKGROUND

The subject matter disclosed herein relates to a handheldthree-dimensional (3D) measurement device, and particularly to a movablescanning system.

A 3D measurement device, also referred to as a 3D triangulation scanneror a 3D imager, is a portable device having a projector that projectslight patterns on the surface of an object to be scanned. One (or more)cameras, having a predetermined positions and alignment relative to theprojector, records images of the light pattern on the surface of anobject. The three-dimensional coordinates of elements in the lightpattern can be determined by trigonometric methods, such as by usingtriangulation. Other types of 3D measuring devices may also be used tomeasure 3D coordinates, such as those that use time of flight techniques(e.g., laser trackers, laser scanners or time of flight cameras) formeasuring the amount of time it takes for light to travel to the surfaceand return to the device.

It is desired to have a handheld 3D measurement device that is easier touse and that gives additional capabilities and performance. Typically,the 3D measurement device is moved during the scanning process, eithercarried by the operator or on a mobile platform. The rate or speed ofmovement impacts the quality of the scanning data. The faster theoperator moves, the less data points may be acquired. While this may beacceptable in some situations, in other areas the lower density of scandata may result in a loss of tracking. Thus, the desired speed at whichthe scanner is moved through the environment may change during thecourse of the scan.

Accordingly, while existing handheld 3D measurement devices are suitablefor their intended purposes, the need for improvement remains,particularly in providing a 3D measurement device that provides feedbackto the user as described herein.

BRIEF DESCRIPTION

According to one aspect of the disclosure, a system is provided. Thesystem comprising a coordinate scanner configured to optically measureand determine a plurality of three-dimensional coordinates to aplurality of locations on at least one surface in the environment, thecoordinate scanner being configured to move through the environmentwhile acquiring the plurality of three-dimensional coordinates. Adisplay having a graphical user interface. One or more processors areprovided that are configured to determine a quality attribute of aprocess of measuring the plurality of three-dimensional coordinatesbased at least in part on the movement of the coordinate scanner in theenvironment and display a graphical quality indicator on the graphicaluser interface based at least in part on the quality attribute, thequality indicator is a graphical element having at least one movableelement.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include the sizeof the movable element being based on the quality attribute. In additionto one or more of the features described herein above, or as analternative, further embodiments of the system may include the at leastone movable element having a plurality of stacked bars in the movableelement. In addition to one or more of the features described hereinabove, or as an alternative, further embodiments of the system mayinclude the movable element having a first bar and a second bar, thefirst bar indicating a speed of the coordinate scanner, the second barindicating an age of the targets.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include thegraphical element further comprising a quality symbol, the at least onechangeable bar indicating a first quality attribute, the quality symbolindicating a second quality attribute. In addition to one or more of thefeatures described herein above, or as an alternative, furtherembodiments of the system may include the first quality attribute beingbased at least in part on the speed of the coordinate scanner, and thesecond quality attribute is based at least in part on a targetattribute.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include the targetattribute being the age of the target. In addition to one or more of thefeatures described herein above, or as an alternative, furtherembodiments of the system may include the graphical element furthercomprising a quality symbol, the one or more processors being furtherconfigured to change the quality symbol based on the quality attribute.In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include theparameter being one of a length of the at least one movable element, acolor of the at least one movable element, or a combination thereof.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include: thegraphical element being a first color when the process of measuring theplurality of three-dimensional coordinates with the coordinate scannerprovides a density of the plurality of three-dimensional coordinatesabove a first threshold; the graphical element being a second color whenthe process of measuring the plurality of three-dimensional coordinateswith the coordinate scanner provides the density of the plurality ofthree-dimensional coordinates below a second threshold; and thegraphical element being a third color when the process of measuring theplurality of three-dimensional coordinates with the coordinate scannerprovides the density of the plurality of three-dimensional coordinatesbeing between the first threshold and the second threshold.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include the firstthreshold and second threshold being based at least in part on thetranslational or rotational speed of movement of the coordinate scanneror an acquisition rate of the plurality of three-dimensionalcoordinates. In addition to one or more of the features described hereinabove, or as an alternative, further embodiments of the system mayinclude the quality attribute being based at least in part on atranslational speed of the coordinate scanner through the environment.In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include thequality attribute being based at least in part on at least one of: anumber of tracking targets; an age of the targets; a rotational speed ofthe coordinate scanner; a quality threshold of images used to track thecoordinate scanner; a number of three-dimensional points acquired; a 3Dgeometry of the environment; a distance to the objects being scanned;and a level of noise in the plurality of three-dimensional coordinates.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include thegraphical user interface having a first portion and a second portion,the quality indicator being positioned in the first portion. In additionto one or more of the features described herein above, or as analternative, further embodiments of the system may include the one ormore processors being configured to display an image of thethree-dimensional coordinates in the second portion. In addition to oneor more of the features described herein above, or as an alternative,further embodiments of the system may include the one or more processorsbeing configured to display a two-dimensional plan view of theenvironment in the second portion. In addition to one or more of thefeatures described herein above, or as an alternative, furtherembodiments of the system may include the one or more processors beingconfigured to determine a trajectory of the coordinate scanner anddisplay the trajectory on the two-dimensional plan view.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include thequality indicator instructs the operator to perform a stationary scan.In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include thequality indicator instructs the operator to perform an anchor scan. Inaddition to one or more of the features described herein above, or as analternative, further embodiments of the system may include the qualityindicator instructs the operator to record an anchor object.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the system may include the one ormore processors being further configured to determine when tracking hasbeen lost, and displaying on the graphical user interface twooverlapping transparent images of the environment. In addition to one ormore of the features described herein above, or as an alternative,further embodiments of the system may include the two overlappingtransparent images having a first transparent image and a secondtransparent image, the first transparent image being a current image ofthe environment acquired by the coordinate scanner, the secondtransparent image being an image from a position and a direction thatthe coordinate scanner has to be moved back to in order to recovertracking.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may include moving acoordinate scanner through an environment, the coordinate scanner beingconfigured to optically measure three-dimensional coordinates; acquiringdetermine a plurality of three-dimensional coordinates to a plurality oflocations on at least one surface in the environment with the coordinatescanner; displaying on a graphical user interface of a display agraphical quality indicator on the graphical user interface based atleast in part on the quality attribute, the quality indicator is agraphical element having at least one movable element.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may include changingthe size of the movable element is based on the quality attribute. Inaddition to one or more of the features described herein above, or as analternative, further embodiments of the method may include the movableelement having a first bar and a second bar, the first bar indicating aspeed of the coordinate scanner, the second bar indicating an age of thetargets.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may include displayingon the graphical user interface a quality symbol, wherein the at leastone changeable element indicates a first quality attribute, the qualitysymbol indicates a second quality attribute. In addition to one or moreof the features described herein above, or as an alternative, furtherembodiments of the method may include determining a speed of thecoordinate scanner wherein the first quality attribute is based at leastin part on the speed of the coordinate scanner, and the second qualityattribute is based at least in part on a target attribute. In additionto one or more of the features described herein above, or as analternative, further embodiments of the method may include the targetattribute being the age of the target.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may include displayinga quality symbol on the graphical user interface and changing thequality symbol based on the quality attribute. In addition to one ormore of the features described herein above, or as an alternative,further embodiments of the method may include the parameter being one ofa length of the at least one movable element, a color of the at leastone movable element, or a combination thereof.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may include: changingthe graphical element to a first color when the process of measuring theplurality of three-dimensional coordinates with the coordinate scannerprovides a density of the plurality of three-dimensional coordinatesabove a first threshold; changing the graphical element to a secondcolor when the process of measuring the plurality of three-dimensionalcoordinates with the coordinate scanner provides the density of theplurality of three-dimensional coordinates below a second threshold; andchanging the graphical element to a third color when the process ofmeasuring the plurality of three-dimensional coordinates with thecoordinate scanner provides the density of the plurality ofthree-dimensional coordinates being between the first threshold and thesecond threshold.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may include the firstthreshold and second threshold being based at least in part on thetranslational or rotational speed of movement of the coordinate scanneror an acquisition rate of the plurality of three-dimensionalcoordinates. In addition to one or more of the features described hereinabove, or as an alternative, further embodiments of the method mayinclude the quality attribute being based at least in part on atranslational speed of the coordinate scanner through the environment.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may include thequality attribute being based at least in part on at least one of: anumber of tracking targets; an age of the targets; a rotational speed ofthe coordinate scanner; a quality threshold of images used to track thecoordinate scanner; a number of three-dimensional points acquired; a 3Dgeometry of the environment; a distance to the objects being scanned;and a level of noise in the plurality of three-dimensional coordinates.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may include thegraphical user interface having a first portion and a second portion,the quality indicator being positioned in the first portion. In additionto one or more of the features described herein above, or as analternative, further embodiments of the method may include displaying animage of the three-dimensional coordinates in the second portion.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may include displayinga two-dimensional plan view of the environment in the second portion. Inaddition to one or more of the features described herein above, or as analternative, further embodiments of the method may include determining atrajectory of the coordinate scanner and display the trajectory on thetwo-dimensional plan view.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may includeinstructing the operator via the quality indicator to perform astationary scan. In addition to one or more of the features describedherein above, or as an alternative, further embodiments of the methodmay include instructing the operator via the quality indicator toperform an anchor scan. In addition to one or more of the featuresdescribed herein above, or as an alternative, further embodiments of themethod may include instructing the operator via the quality indicator torecord an anchor object.

In addition to one or more of the features described herein above, or asan alternative, further embodiments of the method may includedetermining when tracking has been lost, and displaying on the graphicaluser interface two overlapping transparent images of the environment. Inaddition to one or more of the features described herein above, or as analternative, further embodiments of the method may include the twooverlapping transparent images having a first transparent image and asecond transparent image, the first transparent image being a currentimage of the environment acquired by the coordinate scanner, the secondtransparent image being an image from a position and a direction thatthe coordinate scanner has to be moved back to in order to recovertracking.

These and other advantages and features will become more apparent fromthe following description taken in conjunction with the drawings.

BRIEF DESCRIPTION OF DRAWINGS

The subject matter, which is regarded as the disclosure, is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other features, and advantages ofthe disclosure are apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings in which:

FIG. 1 is a schematic illustration of a 3D measurement device accordingto an embodiment of the disclosure;

FIG. 2A is a graphical user interface for the 3D measurement device ofFIG. 1 according to an embodiment of the disclosure;

FIG. 2B, 2C and FIG. 2D are illustrations of a scan quality meteraccording to an embodiment;

FIG. 2E is an illustration of the graphical user interface of FIG. 2Awhen the 3D measurement device loses tracking according to an embodimentof the disclosure;

FIG. 2F is a graphical user interface for the 3D measurement deviceshowing a 2D overview of the scan being performed according to anembodiment of the disclosure;

FIG. 2G is the graphical user interface of FIG. 2F when a stationaryscan is being performed by the 3D measurement device according to anembodiment of the disclosure;

FIG. 2H is the graphical user interface of FIG. 2F when an anchor scanis being performed by the 3D measurement device according to anembodiment of the disclosure;

FIG. 2I is a graphical user interface for the 3D measurement device ofFIG. 1 which allows for assignment of physical buttons on the 3Dmeasurement device according to an embodiment of the disclosure;

FIG. 3 is a front perspective view of a 3D triangulation scanneraccording to an embodiment of the disclosure;

FIG. 4 is a rear perspective view of the 3D triangulation scanneraccording to an embodiment of the disclosure;

FIG. 5A and FIG. 5B are block diagrams of electronics coupled to thetriangulation scanner according to an embodiment of the disclosure;

FIG. 6 illustrates a method of interconnecting a mobile PC with a mobiledisplay using USB tethering according to an embodiment of thedisclosure;

FIG. 7 is a schematic representation of a triangulation scanner having aprojector and a camera according to an embodiment of the disclosure;

FIG. 8A is a schematic representation of a triangulation scanner havinga projector and two cameras according to an embodiment of thedisclosure;

FIG. 8B is a perspective view of a triangulation scanner having aprojector, two triangulation cameras, and a registration cameraaccording to an embodiment of the disclosure;

FIG. 9 is a schematic representation illustrating epipolar terminology;

FIG. 10 is a schematic representation illustrating how epipolarrelations may be advantageously used in when two cameras and a projectorare placed in a triangular shape according to an embodiment of thedisclosure;

FIG. 11 illustrates a system in which 3D coordinates are determined fora grid of uncoded spots projected onto an object according to anembodiment of the disclosure;

FIG. 12 is a perspective view of a mobile scanning platform according toan embodiment;

FIG. 13 is a perspective view of a two-dimensional (2D) scanning andmapping system for use with the mobile scanning platform of FIG. 12 inaccordance with an embodiment;

FIG. 14 is a block diagram of the system of FIG. 13 ;

FIG. 15-17 are schematic illustrations of the operation of system ofFIG. 13 in accordance with an embodiment;

FIG. 18 is a flow diagram of a method of generating a two-dimensionalmap of an environment;

FIGS. 19A-19C are views of a time-of-flight laser scanner for use withthe mobile scanning platform of FIG. 12 in accordance with anembodiment;

FIG. 20 is a flow diagram of a method of scanning an environment usingthe mobile scanning platform of FIG. 12 ;

FIG. 21 is a schematic illustration of a map generated by a mobilescanning platform showing a trajectory.

The detailed description explains embodiments of the disclosure,together with advantages and features, by way of example with referenceto the drawings.

DETAILED DESCRIPTION

Embodiments of the present disclosure provide for a three-dimensional(3D) scanning system with a graphical user interface (GUI) that providesfeedback to the operator of the scanning quality of the scan beingperformed. In an embodiment, the GUI includes a graphical qualityindicator that represents one or more parameters that indicate thequality of the data acquired by the scanner that provides advantages inreducing time to complete a scan and also avoid having the operatorrescanning areas due to a data quality level that is less than desired.In one or more embodiments, the quality of the data is related to themovement of the 3D scanning system.

Referring now to FIG. 1 , an embodiment is shown of a 3D scanning system100. The system 100 is movable as indicated by the arrow 102 andincludes a GUI 104. The graphical user interface 104 may be integralwith, separate from, or distally located relative to the system 100. Ingeneral, the system 100 uses an optical means to emit light 106 towardsa surface 108 and receive light reflected therefrom. The system 100 usesthe light 106 to determine the distance from the system 100 to thesurface 108. When combined with positional information of the system100, the three-dimensional coordinates of the surface 108 may bedetermined.

In an embodiment, the system 100 may be a handheld triangulation orstructured light scanner, such as scanner 300 (FIG. 3 ) for example, ora mobile scanning platform 1200 (FIG. 12 ) for example, generallyreferred to herein as a coordinate scanner or a 3D measurement device.In an embodiment, the mobile scanning platform 1200 includes both atwo-dimensional (2D) scanner and a 3D time-of-flight (TOF) type scanner.It should be appreciated that while certain optical scanning systems maybe described in connection with the GUI 104, this is for exemplarypurposes and the claims should not be so limited. It is contemplatedthat the GUI 104 may be used with any movable 3D scanning system wherethe motion of the system may impact the quality of the scanning dataacquired.

In an embodiment, the system 100 is the same as is described in commonlyowned U.S. patent application Ser. No. 16/806548 entitledThree-Dimensional Measurement Device filed on Mar. 2, 2020, the contentsof which is incorporated by reference in its entirety. In an embodiment,the system 100 is the same as is described in commonly owned UnitedStates Patent Publication 2020/0109943 entitled System and Method ofDefining a Path and Scanning an Environment filed on Sep. 11, 2019, thecontents of which is incorporated by reference in its entirety.

As discussed herein, the movement 102 of the system 100 through theenvironment may effect the quality of the scanning data. It should beappreciated that by moving too quickly, the system 100 will have a lowerdensity of acquired 3D coordinates or points on the surface 108.Depending on the characteristics of the surface 108, at a given speed,the density of points may drop below a threshold. As a result, featureson the surface 108 may not have a desired level of detail or resolution.Further, in some embodiments, the system 100 may determine the positionand pose of the system 100 in 3D space based on the acquiredmeasurements. This determination of position and pose, sometimesreferred to as tracking, is based on the same features being measuredmultiple times as the system 100 is moved. When the system 100 is movedat or greater than a desired speed, the number of features used fortracking may be less than or equal to a threshold, resulting in thesystem 100 not determining its position or pose in the environment,sometimes referred to as losing tracking. In some embodiment, when thesystem 100 loses tracking, the acquired points may not be registered ina common coordinate frame of reference.

In an embodiment, the system 100 includes one or more processors thatoperate the system 100 and determine the plurality of three-dimensionalcoordinates of points on the surface acquired by the system 100 (e.g.the scan data). The one or more processors are further configured todetermine a quality attribute for the scan data. The quality attributemay be determined on a periodic, aperiodic or continuous basis. The oneor more processors are further configured to display a quality indicator110 on the graphical user interface 104.

In an embodiment, the quality attribute may be based at least in part onthe density of the plurality of three-dimensional coordinates. It shouldbe appreciated that the density of the plurality of three-dimensionalcoordinates may be based on the speed the system 100 is moved, or on apoint acquisition rate. In an embodiment, the quality attribute may bebased on an attribute that includes, but is not limited to a number oftracking targets; an age of the targets; a rotational speed of thecoordinate scanner; a quality threshold of images used to track thecoordinate scanner; a number of three-dimensional points acquired; a 3Dgeometry of the environment; a distance to the objects being scanned;and a level of noise in the plurality of three-dimensional coordinates.As used herein, an age of a target is a number of times that a targetappears in successive images by the system 100. It should be appreciatedthat if a target appears in a large number of successive images, thetarget is of higher reliability/quality for purposes of tracking. In anembodiment, it is desirable to have the target appear or be present ingreater than 10 successive images. The targets may be natural features(e.g. edges, corners of surfaces) or artificial markers (e.g.checkerboards, sphere's).

Referring now to FIG. 2A, an embodiment of a graphical user interface200 is shown that may be displayed on the graphical user interface for amobile 3D measurement system, such as system 100 for example. In thisembodiment, the GUI 200 includes a first portion 202 and a secondportion 204. The first portion 202 may include a quality indicator 210,that may be the same as quality indicator 110. In an embodiment, thequality indicator 210 is shaped as an elongated bar or rectangle havinga plurality of lines or gradations 212. The quality indicator 210 mayfurther include a movable element 214. In an embodiment, the movableelement 214 has a parameter, such as a length or size of the element forexample, that changes. As will be discussed in more detail herein, themovable element 214 changes based at least in part on the value of thequality attribute.

Referring now to FIG. 2B-2D, embodiments are shown of differentsequences for the quality indicator 210. In FIG. 2B, the qualityindictor 210 is displayed with the movable element 214 being a firstsize that is small compared to the length of the overall length of theindicator 210. In an embodiment, the movable element 214 may furtherhave an associated first color, such as a green color for example whenthe movable element 214 is the first size. In an embodiment the firstsize corresponds to the movable indicator 214 being a rectangle thatextends between first gradation 212A and second gradation 212B. In anembodiment, the movable element 214 is the first size or first colorwhen the quality attribute is within a range of values (e.g. betweenthresholds) that provides a desirable quality of scan data.

Referring now to FIG. 2C, an embodiment is shown where the size of themovable element 214 has been changed to a second size, such as extendingfrom gradation 212A to gradation 212C for example. In an embodiment, themovable element 214 changes color to a second color, such as yellow forexample, when changed to the second size. In an embodiment, the movableelement 214 is the second size or second color when the qualityattribute is within a second range of values (e.g. between thresholds)that may provide a quality of scan data that is lower than desired.

Referring now to FIG. 2D, an embodiment is shown where the size of themovable element 214 has been changed to a third size, such as extendingfrom gradation 212A to gradation 212D for example. In an embodiment, themovable element 214 changes color to a third color, such as red forexample, when changed to the third size. In an embodiment, the movableelement 214 is the third size or third color when the quality attributeis within a third range of values (e.g. between thresholds) that mayprovide a quality of scan data that is lower than desired. In anembodiment, the displaying of the movable element 214 as the third sizeor third color may indicate that the 3D measurement device 100 may havelost tracking (FIG. 2E) or the loss of tracking is imminent. In anembodiment, when the displaying of the movable element 214 as the thirdsize or third color, an icon 222, such as a triangle with an exclamationpoint for example, may be displayed on GUI 200.

In an embodiment, the icon 222 may be used to indicate a differentquality attribute from the moveable element 214. For example, themoveable element 214 size/color may be based at least in part on thetranslational speed of the system 100, while the icon 222 may be basedon the quality of targets being scanned (e.g. the age of the targets) oron the image quality. In some embodiments, the icon 222 may change basedon the quality attribute that is being indicated. In some embodiments,the quality attribute indicated by either the movable element 214 or theicon 222 may be user selected. In an embodiment, a representation of thescanning device is displayed (FIG. 2B), when the quality attributes arewithin a desired range.

It should be appreciated that while embodiments herein illustrate thefirst portion 202 as having a single movable element 214, this is forexemplary purposes and the claims should not be so limited. In otherembodiments, the first portion may include multiple movable elements,with each movable element representing a different quality element (e.g.speed, point acquisition rate, age of targets, etc.). In still anotherembodiment, the GUI 200 may include a selector that allows the operatorto change the quality attribute indicated by the movable element 214.

Referring to FIG. 2A, in an embodiment, the one or more processors areconfigured to display an image 206 of the scan data in the secondportion 204. It should be appreciated that the displaying of the scandata while the system 100 is being operated provides advantages inallowing the operator to determine whether all of the surfaces orobjects that were desired to be scanned have been acquired. It shouldfurther be appreciated that pairing the display of the scan data withthe quality indicator provides still additional advantages in allowingthe operator to determine what has been scanned and the quality of thatdata.

As discussed, when tracking is lost, the GUI 200 may display the qualityindicator 210 with the movable element 214 being the third length andthird color (e.g. red) as is shown in FIG. 2E. In an embodiment, thedisplay of the second portion 204 also changes to display twooverlapping images. A first image 224 represents a current image of theenvironment and a second image 226 is positioned in a manner to indicateto the operator how to move the system 100 to re-acquire tracking. Inother words, if the operator changes the position and/or pose of thesystem 100 to cause the second image 226 to overlap the first image 224,the system 100 may be able to re-acquire tracking.

In some embodiments, the GUI 200 may have further additional portions,such as a third portion 208 that may include data useful to theoperator, such as an indicator 216 the energy level, an indicator 218 ofthe amount of storage space in memory, and a project name 220.

In some embodiments the GUI 200 may be configured to display a 2D planview 230 of the scan data as is shown in FIG. 2F. In an embodiment, theplan view 230 may include a trajectory 232 or path that the system 100followed in acquiring the scan data. In an embodiment, the plan view 230may further include a symbol 233 or icon indicating where differenttypes of scans were performed along the trajectory 232.

As described in more detail herein, in some embodiments the system 100may be configured to perform different types of scans. These scans maybe manually initiated by the operator, or the user may be prompted bythe one or more processors. In an embodiment, the operator may beprompted to change the type of scan in response to the quality attributebeing in the second range of values (e.g. the quality indicator is thesecond size or second color). To allow the operator to initiate adifferent type of scan, one or more icons 234, 236, 238 or selectors maybe arranged on the GUI 200. For example purposes, the GUI 200 isillustrated with an anchor scan icon 234, a stationary scan icon 238 anda finalize scan icon 236.

An anchor scan is a scan that is performed rapidly (e.g. volume isscanned within 5 seconds) while the system 100 is substantiallystationary. In an embodiment, the anchor scan data is fully integratedwith the scan data from when the system 100 is moving. In embodiments,the anchor scan is used to stabilize the bundling algorithm during postprocessing. In an embodiment, the GUI 200 may display a warning message240 (FIG. 2H) when the anchor scan is being performed.

In still other embodiments, an the operator may be prompted (not shown)by the GUI 200 to record an anchor object. As used herein, an anchorobject is an object in the recorded environment that is used for betteralignment of the point cloud. An anchor object can be for example one ormore natural features (e.g. a corner of a room) or an artificial targetsuch as a marker with a unique ID.

A stationary scan is initiated by selecting the icon 238. The stationaryscan is a scan that is performed at a higher resolution than isperformed when the system 100 is being moved or during an anchor scan.In some embodiments, a stationary scan scans the volume in about aminute. In embodiments, scan data from a stationary scan is stored inmemory separate from the mobile scan data while the system 100 ismoving. This separately sequenced or stored scan data is registered withthe mobile scan data during post processing. In some embodiments,parameters of the system 100 may be adjusted prior to the stationaryscan. In still further embodiments, the stationary scan may acquirecolor data in addition to 3D coordinate data. In an embodiment, the GUI200 may display a warning message 242 (FIG. 2G) when the stationary scanis being performed.

In some embodiments, the GUI 200 may further have a display thatprovides options to the operator without displaying the second portion204. This display may also be the startup or initial display. Referringto FIG. 2I, the options may include a tools element 244 for changingscanner parameters, a settings element 246 for changing parameters ofthe display, and a projects element 248 for allowing the operator toselect how the scan data should be saved. The GUI 200 may furtherinclude operational icons 250, 252, 254 or selectors for controlling theoperation of the system 100. These icons may include a light sourceon/off icon 250, a initiate/stop scan icon 252, and an anchor scan icon254. In an embodiment, the icons or the functionality of the icons 250,252, 254 may be user defined or user selected, such as through thesettings icon 246 for example.

In an embodiment, the icons 250, 252, 254 correspond to physical buttonsor actuators on the system 100. In an embodiment, when the functionalityof the icons 250, 252, 254 is changed, the functionality of the physicalbuttons or actuators on the system 100 are changed to be the same as theicons on GUI 200. In an embodiment, the icons 250, 252, 254 are arrangedin the same manner (e.g. position on the screen) as the physical buttonsor actuators on the system 100.

Referring now to FIGS. 3-11 , an embodiment is shown of the system 100where the 3D measurement device is a hand held triangulation scanner300. FIG. 3 is a front isometric view of a handheld 3D measurementdevice 300, also referred to as a handheld 3D triangulation scanner orimager. In an embodiment, the device 300 includes a first infrared (IR)camera 320, a second IR camera 340, a registration camera 330, aprojector 350, an Ethernet cable 360 and a handle 370. In an embodiment,the registration camera 330 is a color camera. Ethernet is a family ofcomputer networking technologies standardized under IEEE 802.3. Theenclosure 380 includes the outmost enclosing elements of the device 300,as explained in more detail herein below. FIG. 4 is a rear perspectiveview of the device 300 further showing an exemplary perforated rearcover 420 and a scan start/stop button 410. In an embodiment, buttons411, 412 may be programmed to perform functions according to theinstructions of a computer program, the computer program either storedinternally within the device 300 or externally in an external computer.In an embodiment, each of the buttons 410, 411, 412 includes at itsperiphery a ring illuminated by a light emitting diode (LED). In anembodiment, the functionality of buttons 410, 411, 412 may be changed tomatch the icons 250, 252, 254 of FIG. 2I.

In an embodiment, the device 300 of FIG. 3 is the scanner described incommonly owned U.S. patent application Ser. No. 16/806,548 filed on Mar.2, 2020, the contents of which are incorporated by reference herein inits entirety.

FIG. 5A is a block diagram of system electronics 500 that in anembodiment is included in the device 300. In an embodiment, theelectronics 500 includes electronics 510 within the device 300,electronics 570 within the mobile PC 601 (FIG. 6 ), electronics withinthe mobile computing device 603, electronics within other electronicdevices such as accessories that attach to an accessory interface (notshown), and electronics such as external computers that cooperate withthe scanner system electronics 500. In an embodiment, the electronics510 includes a circuit baseboard 512 that includes a sensor collection520 and a computing module 530, which is further shown in FIG. 5B. In anembodiment, the sensor collection 520 includes an IMU and one or moretemperature sensors. In an embodiment, the computing module 530 includesa system-on-a-chip (SoC) field programmable gate array (FPGA) 532. In anembodiment, the SoC FPGA 532 is a Cyclone V SoC FPGA that includes dual800 MHz Cortex A9 cores, which are Advanced RISC Machine (ARM) devices.The Cyclone V SoC FPGA is manufactured by Intel Corporation, withheadquarters in Santa Clara, California. FIG. 5B represents the SoC FPGA532 in block diagram form as including FPGA fabric 534, a Hard ProcessorSystem (HPS) 536, and random access memory (RAM) 538 tied together inthe SoC 539. In an embodiment, the HPS 536 provides peripheral functionssuch as Gigabit Ethernet and USB. In an embodiment, the computing module530 further includes an embedded MultiMedia Card (eMMC) 540 having flashmemory, a clock generator 542, a power supply 544, an FPGA configurationdevice 546, and interface board connectors 548 for electricalcommunication with the rest of the system.

Signals from the infrared (IR) cameras 501A, 501B and the registrationcamera 503 are fed from camera boards through cables to the circuitbaseboard 512. Image signals 552A, 552B, 552C from the cables areprocessed by the computing module 530. In an embodiment, the computingmodule 530 provides a signal 553 that initiates emission of light fromthe laser pointer 505. A TE control circuit communicates with the TEcooler within the infrared laser 509 through a bidirectional signal line554. In an embodiment, the TE control circuit is included within the SoCFPGA 532. In another embodiment, the TE control circuit is a separatecircuit on the baseboard 512. A control line 555 sends a signal to thefan assembly 507 to set the speed of the fans. In an embodiment, thecontrolled speed is based at least in part on the temperature asmeasured by temperature sensors within the sensor unit 520. In anembodiment, the baseboard 512 receives and sends signals to buttons 410,411, 412 and their LEDs through the signal line 556. In an embodiment,the baseboard 512 sends over a line 561 a signal to an illuminationmodule 560 that causes white light from the LEDs to be turned on or off.

In an embodiment, bidirectional communication between the electronics510 and the electronics 570 is enabled by Ethernet communications link565. In an embodiment, the Ethernet link is provided by the cable 360.In an embodiment, the cable 360 attaches to the mobile PC 601 throughthe connector on the bottom of the handle. The Ethernet communicationslink 565 is further operable to provide or transfer power to theelectronics 510 through the user of a custom Power over Ethernet (PoE)module 572 coupled to the battery 574. In an embodiment, the mobile PC570 further includes a PC module 576, which in an embodiment is anIntel® Next Unit of Computing (NUC) processor. The NUC is manufacturedby Intel Corporation, with headquarters in Santa Clara, California. Inan embodiment, the mobile PC 570 is configured to be portable, such asby attaching to a belt and carried around the waist or shoulder of anoperator.

In an embodiment, shown in FIG. 6 , the device 300 may be arranged in afirst configuration 600. In this embodiment, a display 603, such as amobile computing device or cellular phone may be configured tocommunicate with the device 300 or the mobile computing device or mobilePC 601. The communication between the display device 603 and the mobilePC 601 may be by cable or via a wireless medium (e.g. Bluetooth™ orWiFi). In an embodiment, a USB cable connects the mobile phone to thedevice 300, for example, through a USB cable 690 to a compatible USBport on the bottom of the main body of the scanner 10. In an embodiment,using USB tethering, the mobile display 603 is connected to the mobilePC 601 by the Ethernet cable 360 that provides Ethernet link 565.

FIG. 7 shows a 3D measurement device (e.g. triangulation scanner orimager) 700 that projects a pattern of light over an area on a surface730. The device 700, which has a frame of reference 760, includes aprojector 710 and a camera 720. In an embodiment, the projector 710includes an illuminated projector pattern generator 712, a projectorlens 714, and a perspective center 718 through which a ray of light 711emerges. The ray of light 711 emerges from a corrected point 716 havinga corrected position on the pattern generator 712. In an embodiment, thepoint 716 has been corrected to account for aberrations of theprojector, including aberrations of the lens 714, in order to cause theray to pass through the perspective center 718, thereby simplifyingtriangulation calculations. In an embodiment, the pattern generator 712includes a light source that sends a beam of light through a diffractiveoptical element (DOE). For example, the light source might be theinfrared laser 509. A beam of light from the infrared laser 509 passesthrough the DOE, which diffracts the light into a diverging pattern suchas a diverging grid of spots. In an embodiment, one of the projectedrays of light 711 has an angle corresponding to the angle a in FIG. 7 .In another embodiment, the pattern generator 712 includes a light sourceand a digital micromirror device (DMD). In other embodiments, othertypes of pattern generators 712 are used.

The ray of light 711 intersects the surface 730 in a point 732, which isreflected (scattered) off the surface and sent through the camera lens724 to create a clear image of the pattern on the surface 730 of aphotosensitive array 722. The light from the point 732 passes in a ray721 through the camera perspective center 728 to form an image spot atthe corrected point 726. The position of the image spot ismathematically adjusted to correct for aberrations of the camera lens. Acorrespondence is obtained between the point 726 on the photosensitivearray 722 and the point 716 on the illuminated projector patterngenerator 712. As explained herein below, the correspondence may beobtained by using a coded or an uncoded pattern of projected light. Oncethe correspondence is known, the angles a and b in FIG. 7 may bedetermined. The baseline 740, which is a line segment drawn between theperspective centers 718, 728, has a length C. Knowing the angles a, band the length C, all the angles and side lengths of the triangle728-732-718 may be determined. Digital image information is transmittedto a processor 750, which determines 3D coordinates of the surface 730.The processor 750 may also instruct the illuminated pattern generator712 to generate an appropriate pattern.

FIG. 8A shows a structured light 3D measurement device (e.g.triangulation scanner) 800 having a projector 850, a first camera 810,and a second camera 830. The projector 850 creates a pattern of light ona pattern generator 852, which it projects from a corrected point 853 ofthe pattern through a perspective center 858 (point D) of the lens 854onto an object surface 870 at a point 872 (point F). In an embodiment,the pattern generator is a DOE that projects a pattern based onprinciples of diffractive optics. In other embodiments, other types ofpattern generators are used. The point 872 is imaged by the first camera810 by receiving a ray of light from the point 872 through a perspectivecenter 818 (point E) of a lens 814 onto the surface of a photosensitivearray 812 of the camera as a corrected point 820. The point 820 iscorrected in the read-out data by applying a correction factor to removethe effects of lens aberrations. The point 872 is likewise imaged by thesecond camera 830 by receiving a ray of light from the point 872 througha perspective center 838 (point C) of the lens 834 onto the surface of aphotosensitive array 832 of the second camera as a corrected point 835.It should be understood that any reference to a lens in this document isunderstood to mean any possible combination of lens elements andapertures.

FIG. 8B shows 3D measurement device (e.g. imager) 880 having two cameras881, 883 and a projector 885 arranged in a triangle A₁-A₂-A₃. In anembodiment, the device 880 of FIG. 8B further includes a camera 889 thatmay be used to provide color (texture) information for incorporationinto the 3D image. In addition, the camera 889 may be used to registermultiple 3D images through the use of videogrammetry. This triangulararrangement provides additional information beyond that available fortwo cameras and a projector arranged in a straight line as illustratedin FIG. 8A. The additional information may be understood in reference toFIG. 8 , which explains the concept of epipolar constraints, and FIG. 10, which explains how epipolar constraints are advantageously applied tothe triangular arrangement of the device 880. In an embodiment, theelements 881, 883, 885, 889 in FIG. 8B correspond to the elements 340,320, 350, 330 in FIG. 3 .

In FIG. 9 , a 3D measurement device (e.g. triangulation instrument) 940includes a device 1 and a device 2 on the left and right sides,respectively. Device 1 and device 2 may be two cameras or device 1 anddevice 2 may be one camera and one projector. Each of the two devices,whether a camera or a projector, has a perspective center, O₁ and O₂,and a reference plane, 930 or 910. The perspective centers are separatedby a baseline distance B, which is the length of the line 902 between O₁and O₂. The perspective centers O₁, O₂ are points through which rays oflight may be considered to travel, either to or from a point on anobject. These rays of light either emerge from an illuminated projectorpattern or impinge on a photosensitive array.

In FIG. 9 , a device 1 has a perspective center O₁ and a reference plane930, where the reference plane 930 is, for the purpose of discussion,equivalent to an image plane of the object point O₁ 930. In other words,the reference plane 930 is a projection of the image plane about theperspective center O₁. A device 2 has a perspective center O₂ and areference plane 910. A line 902 drawn between the perspective centers O₁and O₂ crosses the planes 930 and 910 at the epipole points E₁, E₂,respectively. Consider a point U_(D) on the plane 930. If device 1 is acamera, an object point that produces the point U_(D) on the referenceplane 930 (which is equivalent to a corresponding point on the image)must lie on the line 938. The object point might be, for example, one ofthe points V_(A), V_(B), V_(C), or V_(D). These four object pointscorrespond to the points W_(A), W_(B), W_(C), W_(D), respectively, onthe reference plane 910 of device 2. This is true whether device 2 is acamera or a projector. It is also true that the four points lie on astraight line 912 in the plane 910. This line, which is the line ofintersection of the reference plane 910 with the plane of O₁-O₂-U_(D),is referred to as the epipolar line 912. It follows that any epipolarline on the reference plane 910 passes through the epipole E₂. Just asthere is an epipolar line on the reference plane 910 of device 2 for anypoint U_(D) on the reference plane of device 1, there is also anepipolar line 934 on the reference plane 930 of device 1 for any pointon the reference plane 910 of device 2.

FIG. 10 illustrates the epipolar relationships for a 3D measurementdevice (e.g. imager) 1090 corresponding to device 880 of FIG. 8B inwhich two cameras and one projector are arranged in a triangularpattern. In general, the device 1, device 2, and device 3 may be anycombination of cameras and projectors as long as at least one of thedevices is a camera. Each of the three devices 1091, 1092, 1093 has aperspective center O₁, O₂, O₃, respectively, and a reference plane 1060,1070, and 1080, respectively. Each pair of devices has a pair ofepipoles. Device 1 and device 2 have epipoles E₁₂, E₂₁ on the planes1060, 1070, respectively. Device 1 and device 3 have epipoles E₁₃, E₃₁,respectively on the planes 1060, 1080, respectively. Device 2 and device3 have epipoles E₂₃, E₃₂ on the planes 1070, 1080, respectively. Inother words, each reference plane includes two epipoles. The referenceplane for device 1 includes epipoles E₁₂ and E₁₃. The reference planefor device 2 includes epipoles E₂₁ and E₂₃. The reference plane fordevice 3 includes epipoles E₃₁ and E₃₂.

Consider the embodiment of FIG. 10 in which device 3 is a projector,device 1 is a first camera, and device 2 is a second camera. Supposethat a projection point P₃, a first image point P₁, and a second imagepoint P₂ are obtained in a measurement. These results can be checked forconsistency in the following way.

To check the consistency of the image point P₁, intersect the planeP₃-E₃₁-E₁₃ with the reference plane 1060 to obtain the epipolar line1064. Intersect the plane P₂-E₂₁-E₁₂ to obtain the epipolar line 1062.If the image point P₁ has been determined consistently, the observedimage point P₁ will lie on the intersection of the calculated epipolarlines 1062 and 1064.

To check the consistency of the image point P₂, intersect the planeP₃-E₃₂-E₂₃ with the reference plane 1070 to obtain the epipolar line1074. Intersect the plane P₁-E₁₂-E₂₁ to obtain the epipolar line 1072.If the image point P₂ has been determined consistently, the observedimage point P₂ will lie on the intersection of the calculated epipolarline 1072 and epipolar line 1074.

To check the consistency of the projection point P₃, intersect the planeP₂-E₂₃-E₃₂ with the reference plane 1080 to obtain the epipolar line1084. Intersect the plane P₁-E₁₃-E₃₁ to obtain the epipolar line 1082.If the projection point P₃ has been determined consistently, theprojection point P₃ will lie on the intersection of the calculatedepipolar lines 1082, 1084.

The redundancy of information provided by using a 3D imager having threedevices (such as two cameras and one projector) enables a correspondenceamong projected points to be established even without analyzing thedetails of the captured images and projected pattern features. Suppose,for example, that the three devices include two cameras and oneprojector. Then a correspondence among projected and imaged points maybe directly determined based on the mathematical constraints of theepipolar geometry. This may be seen in FIG. 10 by noting that a knownposition of an illuminated point on one of the reference planes 1060,1070, 1080 automatically provides the information needed to determinethe location of that point on the other two reference planes.Furthermore, once a correspondence among points has been determined oneach of the three reference planes 1060, 1070, 1080, a triangulationcalculation may be performed using only two of the three devices of FIG.10 . A description of such a triangulation calculation is discussed inrelation to FIG. 9 .

By establishing correspondence based on epipolar constraints, it ispossible to determine 3D coordinates of an object surface by projectinguncoded spots of light. An example of projection of uncoded spots isillustrated in FIG. 11 . In an embodiment, a projector 1110 projects acollection of identical spots of light 1121 on an object 1120. In theexample shown, the surface of the object 1120 is curved in an irregularmanner causing an irregular spacing of the projected spots on thesurface. One of the projected points is the point 1122, projected from aprojector source element 1112 and passing through the perspective center1116 as a ray of light 1124 forms a point 1118 on the reference plane1114.

The point or spot of light 1122 on the object 1120 is projected as a rayof light 1126 through the perspective center 1132 of a first camera1130, resulting in a point 1134 on the image sensor of the camera 1130.The corresponding point 1138 is located on the reference plane 1136.Likewise, the point or spot of light 1122 is projected as a ray of light1128 through the perspective center 1142 of a second camera 1140,resulting in a point 1144 on the image sensor of the camera 1140. Thecorresponding point 1148 is located on the reference plane 1146. In anembodiment, a processor 1150 is in communication with the projector1110, first camera 1130, and second camera 1140. The processordetermines a correspondence among points on the projector 1110, firstcamera 1130, and second camera 1140. In an embodiment, the processor1150 performs a triangulation calculation to determine the 3Dcoordinates of the point 1122 on the object 1120. An advantage of ascanner 1100 having three device elements, either two cameras and oneprojector or one camera and two projectors, is that correspondence maybe determined among projected points without matching projected featurecharacteristics. In other words, correspondence can be established amongspots on the reference planes 1136, 1114, and 1146 even without matchingparticular characteristics of the spots. The use of the three devices1110, 1130, 1140 also has the advantage of enabling identifying orcorrecting errors in compensation parameters by noting or determininginconsistencies in results obtained from triangulation calculations, forexample, between two cameras, between the first camera and theprojector, and between the second camera and the projector.

Referring now to FIGS. 13-21 , an embodiment is shown of system 100having a mobile scanning platform 1200. The platform 1200 includes aframe 1202 having a tripod portion 1204 thereon. The frame 1202 furtherincludes a plurality of wheels 1206 that allow the platform 1200 to bemoved about an environment. The frame 1202 further includes a handleportion 1207 that provides a convenient place for the operator to pushand maneuver the platform 1200.

The tripod portion 1204 includes a center post 1209. In an embodiment,the center post 1209 generally extends generally perpendicular to thesurface that the platform 1200 is on. Coupled to the top of the post1209 is a 3D measurement device 1210. In the exemplary embodiment, the3D measurement device 1210 is a time-of-flight type scanner (eitherphase-based or pulse-based) that emits and receives a light to measure avolume about the scanner. In the exemplary embodiment, the 3Dmeasurement device 1210 is the same as that described in reference toFIGS. 19A-19C herein.

Also attached to the center post 1209 is a 2D scanner 1308. In anembodiment, the 2D scanner 1308 is the same type of scanner as isdescribed in reference to FIGS. 13-18 herein. In the exemplaryembodiment, the 2D scanner emits light in a plane and measures adistance to an object, such as a wall for example. As described in moredetail herein, these distance measurements may be used to generate a 2Dmap of an environment when the 2D scanner 1308 is moved therethrough.The 2D scanner 1308 is coupled to the center post by an arm thatincludes an opening to engage at least the handle portion of the 2Dscanner 1308.

In an embodiment, one or both of the 3D scanner 1210 and the 2D scanner1308 are removably coupled from the platform 1200. In an embodiment, theplatform 1200 is configured to operate (e.g. operate the scanners 1308,1210) while the platform 1200 is being carried by one or more operators.

In an embodiment, the mobile scanning platform 1200 may include acontroller (not shown) that is coupled to communicate with both the 2Dscanner 1308 and the 3D measurement device 1210.

Is should be appreciated that the platform 1200 is manually pushed by anoperator through the environment. As will be discussed in more detailherein, as the platform 1200 is moved through the environment, both the2D scanner 1308 and the 3D measurement device 1210 are operatedsimultaneously, with the data of the 2D measurement device being used,at least in part, to register the data of the 3D measurement system.

If should further be appreciated that in some embodiments, it may bedesired to the measurement platform 1200 to be motorized in asemi-autonomous or fully-autonomous configuration. It should beappreciated that in an embodiment where the measurement platform 1200 isin a semi-autonomous or fully-autonomous configuration, the qualityattribute may be a feedback signal to the one or more processors. Inthis embodiment, the one or more processors may automatically adjust thespeed and/or direction of the

Referring now to FIGS. 13-18 , an embodiment of a 2D scanner 1308 isshown having a housing 1332 that includes a body portion 1334 and aremovable handle portion 1336. It should be appreciated that while theembodiment of FIGS. 13-18 illustrate the 2D scanner 1308 with the handle1336 attached, the handle 1336 may be removed before the 2D scanner 1308is coupled to the platform 1200. In an embodiment, the handle 1336 mayinclude an actuator that allows the operator to interact with thescanner 1308. In the exemplary embodiment, the body 1334 includes agenerally rectangular center portion 1335 with a slot 1340 formed in anend 1342. The slot 1340 is at least partially defined by a pair walls1344 that are angled towards a second end. A portion of a 2D laserscanner 1350 is arranged between the walls 1344. The walls 1344 areangled to allow the 2D laser scanner 1350 to operate by emitting a lightover a large angular area without interference from the walls 1344. Aswill be discussed in more detail herein, the end 1342 may furtherinclude a three-dimensional camera or RGBD camera.

Extending from the center portion 1335 is a mobile device holder 1341.The mobile device holder 1341 is configured to securely couple a mobiledevice 1343 to the housing 1332. The holder 1341 may include one or morefastening elements, such as a magnetic or mechanical latching elementfor example, that couples the mobile device 1343 to the housing 1332. Inan embodiment, the mobile device 1343 is coupled to communicate with acontroller 1368 (FIG. 14 ). The communication between the controller1368 and the mobile device 1343 may be via any suitable communicationsmedium, such as wired, wireless or optical communication mediums forexample.

In the illustrated embodiment, the holder 1341 is pivotally coupled tothe housing 1332, such that it may be selectively rotated into a closedposition within a recess. In an embodiment, the recess is sized andshaped to receive the holder 1341 with the mobile device 1343 disposedtherein. It should further be appreciated that when the 2D scanner 1308is coupled to the mobile platform 1200 and a control system for themobile platform 1200 (that may include one or more processors), then theoperation of the 2D scanner 1308 may be controlled by the mobileplatform controller.

In the exemplary embodiment, the second end 1348 includes a plurality ofexhaust vent openings 1356. In an embodiment, the exhaust vent openings1356 are fluidly coupled to intake vent openings arranged on a bottomsurface of center portion 1335. The intake vent openings allow externalair to enter a conduit having an opposite opening in fluid communicationwith the hollow interior of the body 1334. In an embodiment, the openingis arranged adjacent to the controller 1368 which has one or moreprocessors that is operable to perform the methods described herein. Inan embodiment, the external air flows from the opening over or aroundthe controller 1368 and out the exhaust vent openings 1356.

The controller 1368 is electrically coupled to the 2D laser scanner1350, the 3D camera 1360, a power source 1372, an inertial measurementunit (IMU) 1374, a laser line projector 1376, and a haptic feedbackdevice 1377 (FIG. 14 ).

Referring now to FIG. 14 with continuing reference to FIGS. 12-13 ,elements are shown of the scanner 1308 with the mobile device 1343installed or coupled to the housing 1332. It should be appreciated thatin some embodiments, the functionality of the mobile device 1343 may bereplaced by the controller for the mobile platform 1200. Controller 1368is a suitable electronic device capable of accepting data andinstructions, executing the instructions to process the data, andpresenting the results. The controller 1368 includes one or moreprocessing elements 1378. The processors may be microprocessors, fieldprogrammable gate arrays (FPGAs), digital signal processors (DSPs), andgenerally any device capable of performing computing functions. The oneor more processors 1378 have access to memory 1380 for storinginformation.

Controller 1368 is capable of converting the analog voltage or currentlevel provided by 2D laser scanner 1350, camera 1360 and IMU 1374 into adigital signal to determine a distance from the scanner 1308 to anobject in the environment. In an embodiment, the camera 1360 is a 3D orRGBD type camera. Controller 1368 uses the digital signals that act asinput to various processes for controlling the scanner 1308. The digitalsignals represent one or more scanner 1308 data including but notlimited to distance to an object, images of the environment,acceleration, pitch orientation, yaw orientation and roll orientation.As will be discussed in more detail, the digital signals may be fromcomponents internal to the housing 1332 or from sensors and deviceslocated in the mobile device 1343.

In general, when the mobile device 1343 is not installed, controller1368 accepts data from 2D laser scanner 1350 and IMU 1374 and is givencertain instructions for the purpose of generating a two-dimensional mapof a scanned environment. Controller 1368 provides operating signals tothe 2D laser scanner 1350, the camera 1360, laser line projector 1376and haptic feedback device 1377. Controller 1368 also accepts data fromIMU 1374, indicating, for example, whether the operator is operating inthe system in the desired orientation. In an embodiment, the controller1368 compares the operational parameters to predetermined variances(e.g. yaw, pitch or roll thresholds) and if the predetermined varianceis exceeded, generates a signal that activates the haptic feedbackdevice 1377. The data received by the controller 1368 may be displayedon a user interface coupled to controller 1368. The user interface maybe one or more LEDs (light-emitting diodes) 1382, an LCD (liquid-crystaldiode) display, a CRT (cathode ray tube) display, or the like. A keypadmay also be coupled to the user interface for providing data input tocontroller 1368. In one embodiment, the user interface is arranged orexecuted on the mobile device 1343.

The controller 1368 may also be coupled to external computer networkssuch as a local area network (LAN) and the Internet. A LAN interconnectsone or more remote computers, which are configured to communicate withcontrollers 1368 using a well- known computer communications protocolsuch as TCP/IP (Transmission Control Protocol/Internet({circumflex over( )}) Protocol), RS-232, ModBus, and the like. additional scanners 1308may also be connected to LAN with the controllers 1368 in each of thesescanners 1308 being configured to send and receive data to and fromremote computers and other scanners 1308. The LAN may be connected tothe Internet. This connection allows controller 1368 to communicate withone or more remote computers connected to the Internet.

The processors 1378 are coupled to memory 1380. The memory 1380 mayinclude random access memory (RAM) device 1384, a non-volatile memory(NVM) device 1386, a read-only memory (ROM) device 1388. In addition,the processors 1378 may be connected to one or more input/output (I/O)controllers 1390 and a communications circuit 1392. In an embodiment,the communications circuit 1392 provides an interface that allowswireless or wired communication with one or more external devices ornetworks, such as the LAN discussed above or the communications circuit1318.

Controller 1368 includes operation control methods embodied inapplication code such as that shown or described with reference to FIGS.15-18 . These methods are embodied in computer instructions written tobe executed by processors 1378, typically in the form of software. Thesoftware can be encoded in any language, including, but not limited to,assembly language, VHDL (Verilog Hardware Description Language), VHSICHDL (Very High Speed IC Hardware Description Language), Fortran (formulatranslation), C, C++, C#, Objective-C, Visual C++, Java, ALGOL(algorithmic language), BASIC (beginners all-purpose symbolicinstruction code), visual BASIC, ActiveX, HTML (HyperText MarkupLanguage), Python, Ruby and any combination or derivative of at leastone of the foregoing.

Coupled to the controller 1368 is the 2D laser scanner 1350. The 2Dlaser scanner 1350 measures 2D coordinates in a plane. In the exemplaryembodiment, the scanning is performed by steering light within a planeto illuminate object points in the environment. The 2D laser scanner1350 collects the reflected (scattered) light from the object points todetermine 2D coordinates of the object points in the 2D plane. In anembodiment, the 2D laser scanner 1350 scans a spot of light over anangle while at the same time measuring an angle value and correspondingdistance value to each of the illuminated object points.

Examples of 2D laser scanners 1350 include, but are not limited to ModelLMS100 scanners manufactured by Sick, Inc of Minneapolis, Minn. andscanner Models URG-04LX-UGO1 and UTM-30LX manufactured by HokuyoAutomatic Co., Ltd of Osaka, Japan. The scanners in the Sick LMS100family measure angles over a 270 degree range and over distances up to20 meters. The Hoyuko model URG-04LX-UG01 is a low-cost 2D scanner thatmeasures angles over a 240 degree range and distances up to 4 meters.The Hoyuko model UTM-30LX is a 2D scanner that measures angles over a270 degree range and to distances up to 30 meters. It should beappreciated that the above 2D scanners are exemplary and other types of2D scanners are also available.

In an embodiment, the 2D laser scanner 1350 is oriented so as to scan abeam of light over a range of angles in a generally horizontal plane(relative to the floor of the environment being scanned). At instants intime the 2D laser scanner 1350 returns an angle reading and acorresponding distance reading to provide 2D coordinates of objectpoints in the horizontal plane. In completing one scan over the fullrange of angles, the 2D laser scanner returns a collection of pairedangle and distance readings. As the platform 1200 is moved from place toplace, the 2D laser scanner 1350 continues to return 2D coordinatevalues. These 2D coordinate values are used to locate the position ofthe scanner 1308 thereby enabling the generation of a two-dimensionalmap or floorplan of the environment.

Also coupled to the controller 1386 is the IMU 1374. The IMU 1374 is aposition/orientation sensor that may include accelerometers 1394(inclinometers), gyroscopes 1396, a magnetometers or compass 1398, andaltimeters. In the exemplary embodiment, the IMU 1374 includes multipleaccelerometers 1394 and gyroscopes 1396. The compass 1398 indicates aheading based on changes in magnetic field direction relative to theearth's magnetic north. The IMU 1374 may further have an altimeter thatindicates altitude (height). An example of a widely used altimeter is apressure sensor. By combining readings from a combination ofposition/orientation sensors with a fusion algorithm that may include aKalman filter, relatively accurate position and orientation measurementscan be obtained using relatively low-cost sensor devices. In theexemplary embodiment, the IMU 1374 determines the pose or orientation ofthe scanner 1308 about three-axis to allow a determination of a yaw,roll and pitch parameter.

In an embodiment, the scanner 1308 further includes a camera 1360 thatis a 3D or RGB-D camera. As used herein, the term 3D camera refers to adevice that produces a two-dimensional image that includes distances toa point in the environment from the location of scanner 1308. The 3Dcamera 1360 may be a range camera or a stereo camera. In an embodiment,the 3D camera 1360 includes an RGB-D sensor that combines colorinformation with a per-pixel depth information. In an embodiment, the 3Dcamera 1360 may include an infrared laser projector 1331 (FIG. 17 ), afirst infrared camera 1333, a second infrared camera 1339, and a colorcamera 1337. In an embodiment, the 3D camera 460 is a RealSense™ cameramodel R200 manufactured by Intel Corporation.

In an embodiment, when the mobile device 1343 is coupled to the housing1332, the mobile device 1343 becomes an integral part of the scanner1308. In an embodiment, the mobile device 1343 is a cellular phone, atablet computer or a personal digital assistant (PDA). The mobile device1343 may be coupled for communication via a wired connection, such asports 1400, 1402. The port 1400 is coupled for communication to theprocessor 1378, such as via I/O controller 1390 for example. The ports1400, 1402 may be any suitable port, such as but not limited to USB,USB-A, USB-B, USB-C, IEEE 1394 (Firewire), or Lightning™ connectors.

The mobile device 1343 is a suitable electronic device capable ofaccepting data and instructions, executing the instructions to processthe data, and presenting the results. The mobile device 1343 includesone or more processing elements 1404. The processors may bemicroprocessors, field programmable gate arrays (FPGAs), digital signalprocessors (DSPs), and generally any device capable of performingcomputing functions. The one or more processors 1404 have access tomemory 1406 for storing information.

The mobile device 1343 is capable of converting the analog voltage orcurrent level provided by sensors 1408 and processor 1378. Mobile device1343 uses the digital signals that act as input to various processes forcontrolling the scanner 1308. The digital signals represent one or moreplatform 1200 data including but not limited to distance to an object,images of the environment, acceleration, pitch orientation, yaworientation, roll orientation, global position, ambient light levels,and altitude for example.

In general, mobile device 1343 accepts data from sensors 1408 and isgiven certain instructions for the purpose of generating or assistingthe processor 1378 in the generation of a two-dimensional map orthree-dimensional map of a scanned environment. Mobile device 1343provides operating signals to the processor 1378, the sensors 1408 and adisplay 1410. Mobile device 1343 also accepts data from sensors 1408,indicating, for example, to track the position of the mobile device 1343in the environment or measure coordinates of points on surfaces in theenvironment. The mobile device 1343 compares the operational parametersto predetermined variances (e.g. yaw, pitch or roll thresholds) and ifthe predetermined variance is exceeded, may generate a signal. The datareceived by the mobile device 1343 may be displayed on display 1410. Inan embodiment, the display 1410 is a touch screen device that allows theoperator to input data or control the operation of the scanner 1308.

The controller 368 may also be coupled to external networks such as alocal area network (LAN), a cellular network and the Internet. A LANinterconnects one or more remote computers, which are configured tocommunicate with controller 68 using a well- known computercommunications protocol such as TCP/IP (Transmission ControlProtocol/Internet({circumflex over ( )}) Protocol), RS-232, ModBus, andthe like. additional scanners 1308 may also be connected to LAN with thecontrollers 1368 in each of these scanners 1308 being configured to sendand receive data to and from remote computers and other scanners 1308.The LAN may be connected to the Internet. This connection allowscontroller 1368 to communicate with one or more remote computersconnected to the Internet.

The processors 1404 are coupled to memory 1406. The memory 1406 mayinclude random access memory (RAM) device, a non-volatile memory (NVM)device, and a read-only memory (ROM) device. In addition, the processors1404 may be connected to one or more input/output (I/O) controllers 1412and a communications circuit 1414. In an embodiment, the communicationscircuit 1414 provides an interface that allows wireless or wiredcommunication with one or more external devices or networks, such as theLAN or the cellular network discussed above.

Controller 1368 includes operation control methods embodied inapplication code shown or described with reference to FIGS. 15-18 .These methods are embodied in computer instructions written to beexecuted by processors 1378, 1404, typically in the form of software.The software can be encoded in any language, including, but not limitedto, assembly language, VHDL (Verilog Hardware Description Language),VHSIC HDL (Very High Speed IC Hardware Description Language), Fortran(formula translation), C, C++, C#, Objective-C, Visual C++, Java, ALGOL(algorithmic language), BASIC (beginners all-purpose symbolicinstruction code), visual BASIC, ActiveX, HTML (HyperText MarkupLanguage), Python, Ruby and any combination or derivative of at leastone of the foregoing.

Also coupled to the processor 1404 are the sensors 1408. The sensors1408 may include but are not limited to: a microphone 1416; a speaker1418; a front or rear facing camera 1420; accelerometers 1422(inclinometers), gyroscopes 1424, a magnetometers or compass 1426; aglobal positioning satellite (GPS) module 1428; a barometer 1430; aproximity sensor 1432; and an ambient light sensor 1434. By combiningreadings from a combination of sensors 1408 with a fusion algorithm thatmay include a Kalman filter, relatively accurate position andorientation measurements can be obtained.

It should be appreciated that the sensors 1360, 1374 integrated into thescanner 1308 may have different characteristics than the sensors 1408 ofmobile device 1343. For example, the resolution of the cameras 1360,1420 may be different, or the accelerometers 1394, 1422 may havedifferent dynamic ranges, frequency response, sensitivity (mV/g) ortemperature parameters (sensitivity or range). Similarly, the gyroscopes1396, 1424 or compass/magnetometer may have different characteristics.It is anticipated that in some embodiments, one or more sensors 1408 inthe mobile device 1343 may be of higher accuracy than the correspondingsensors 1374 in the scanner 1308. As described in more detail herein, insome embodiments the processor 1378 determines the characteristics ofeach of the sensors 1408 and compares them with the correspondingsensors in the scanner 1308 when the mobile device. The processor 1378then selects which sensors 1374, 1408 are used during operation. In someembodiments, the mobile device 1343 may have additional sensors (e.g.microphone 1416, camera 1420) that may be used to enhance operationcompared to operation of the scanner 1308 without the mobile device1343. In still further embodiments, the scanner 1308 does not includethe IMU 1374 and the processor 1378 uses the sensors 1408 for trackingthe position and orientation/pose of the scanner 1308. In still furtherembodiments, the addition of the mobile device 1343 allows the scanner1308 to utilize the camera 1420 to perform three-dimensional (3D)measurements either directly (using an RGB-D camera) or usingphotogrammetry techniques to generate 3D maps. In an embodiment, theprocessor 1378 uses the communications circuit (e.g. a cellular 4Ginternet connection) to transmit and receive data from remote computersor devices.

In an embodiment, the scanner 1308 determines a qualityattribute/parameter for the tracking of the scanner 1308 and/or theplatform 1200. In an embodiment, the tracking quality attribute is aconfidence level in the determined tracking positions and orientationsto actual positions and orientations. When the confidence level crossesa threshold, the platform 1200 may provide feedback to the operator toperform a stationary scan. It should be appreciated that a stationaryscan will provide a highly accurate measurements that will allow thedetermination of the position and orientation of the scanner or platformwith a high level of confidence. In an embodiment, the feedback isprovided via a user interface. The user interface may be on the platform1200, the scanner 1308, or the scanner 1510 for example.

In the exemplary embodiment, the scanner 1308 is a handheld portabledevice that is sized and weighted to be carried by a single personduring operation. Therefore, the plane in which the 2D laser scanner 450projects a light beam may not be horizontal relative to the floor or maycontinuously change as the computer moves during the scanning process.Thus, the signals generated by the accelerometers 1394, gyroscopes 1396and compass 1398 (or the corresponding sensors 1408) may be used todetermine the pose (yaw, roll, tilt) of the scanner 1308 and determinethe orientation of the plane 1351.

In an embodiment, it may be desired to maintain the pose of the scanner1308 (and thus the plane 1436) within predetermined thresholds relativeto the yaw, roll and pitch orientations of the scanner 1308. In anembodiment, a haptic feedback device 1377 is disposed within the housing1332, such as in the handle 1336. The haptic feedback device 1377 is adevice that creates a force, vibration or motion that is felt or heardby the operator. The haptic feedback device 1377 may be, but is notlimited to: an eccentric rotating mass vibration motor or a linearresonant actuator for example. The haptic feedback device is used toalert the operator that the orientation of the light beam from 2D laserscanner 1350 is equal to or beyond a predetermined threshold. Inoperation, when the IMU 1374 measures an angle (yaw, roll, pitch or acombination thereof), the controller 1368 transmits a signal to a motorcontroller 1438 that activates a vibration motor 1440. Since thevibration originates in the handle 1336, the operator will be notifiedof the deviation in the orientation of the scanner 1308. The vibrationcontinues until the scanner 1308 is oriented within the predeterminedthreshold or the operator releases the actuator 1338. In an embodiment,it is desired for the plane 1436 to be within 10-15 degrees ofhorizontal (relative to the ground) about the yaw, roll and pitch axes.

In an embodiment, the 2D laser scanner 1350 makes measurements as theplatform 1200 is moved about an environment, such from a first position1442 to a second registration position 1444 as shown in FIG. 15 . In anembodiment, 2D scan data is collected and processed as the scanner 1308passes through a plurality of 2D measuring positions 1446. At eachmeasuring position 1446, the 2D laser scanner 1450 collects 2Dcoordinate data over an effective FOV 1448. Using methods described inmore detail below, the controller 1368 uses 2D scan data from theplurality of 2D scans at positions 1446 to determine a position andorientation of the scanner 1308 as it is moved about the environment. Inan embodiment, the common coordinate system is represented by 2DCartesian coordinates x, y and by an angle of rotation θ relative to thex or y axis. In an embodiment, the x and y axes lie in the plane of the2D scanner and may be further based on a direction of a “front” of the2D laser scanner 1350.

FIG. 17 shows the 2D scanner 1308 collecting 2D scan data at selectedpositions 1446 over an effective FOV 1448. At different positions 1446,the 2D laser scanner 1350 captures a portion of the object 1450 markedA, B, C, D, and E (FIG. 16 ). FIG. 17 shows 2D laser scanner 1350 movingin time relative to a fixed frame of reference of the object 1450.

FIG. 17 includes the same information as FIG. 16 but shows it from theframe of reference of the scanner 1308 rather than the frame ofreference of the object 1450. FIG. 17 illustrates that in the scanner1308 frame of reference, the position of features on the object changeover time. Therefore, the distance traveled by the scanner 1308 can bedetermined from the 2D scan data sent from the 2D laser scanner 1350 tothe controller 1368.

As the 2D laser scanner 1350 takes successive 2D readings and performsbest-fit calculations, the controller 1368 keeps track of thetranslation and rotation of the 2D laser scanner 1350, which is the sameas the translation and rotation of the scanner 1308. In this way, thecontroller 1368 is able to accurately determine the change in the valuesof x, y, θ as the scanner 1308 moves from the first position 1442 to thesecond position 1444.

In an embodiment, the controller 1368 is configured to determine a firsttranslation value, a second translation value, along with first andsecond rotation values (yaw, roll, pitch) that, when applied to acombination of the first 2D scan data and second 2D scan data, resultsin transformed first 2D data that closely matches transformed second 2Ddata according to an objective mathematical criterion. In general, thetranslation and rotation may be applied to the first scan data, thesecond scan data, or to a combination of the two. For example, atranslation applied to the first data set is equivalent to a negative ofthe translation applied to the second data set in the sense that bothactions produce the same match in the transformed data sets. An exampleof an “objective mathematical criterion” is that of minimizing the sumof squared residual errors for those portions of the scan datadetermined to overlap. Another type of objective mathematical criterionmay involve a matching of multiple features identified on the object.For example, such features might be the edge transitions 1452, 1454, and1456 shown in FIG. 15 . The mathematical criterion may involveprocessing of the raw data provided by the 2D laser scanner 1350 to thecontroller 1368, or it may involve a first intermediate level ofprocessing in which features are represented as a collection of linesegments using methods that are known in the art, for example, methodsbased on the Iterative Closest Point (ICP). Such a method based on ICPis described in Censi, A., “An ICP variant using a point-to-linemetric,” IEEE International Conference on Robotics and Automation (ICRA)2008, which is incorporated by reference herein.

In an embodiment, assuming that the plane of the light beam from 2Dlaser scanner 1350 remains horizontal relative to the ground plane, thefirst translation value is dx, the second translation value is dy, andthe first rotation value dθ. If the first scan data is collected withthe 2D laser scanner 1350 having translational and rotationalcoordinates (in a reference coordinate system) of (x₁, y₁, θ₁), thenwhen the second 2D scan data is collected at a second location thecoordinates are given by (x₂, y₂, θ₂)=(x₁+dx, y₁+dy, θ₁+dθ). In anembodiment, the controller 468 is further configured to determine athird translation value (for example, dz) and a second and thirdrotation values (for example, pitch and roll). The third translationvalue, second rotation value, and third rotation value may be determinedbased at least in part on readings from the IMU 1374.

The 2D laser scanner 1350 collects 2D scan data starting at the firstposition 1442 and more 2D scan data at the second position 1444. In somecases, these scans may suffice to determine the position and orientationof the scanner 1308 at the second position 1444 relative to the firstposition 1442. In other cases, the two sets of 2D scan data are notsufficient to enable the controller 1368 to accurately determine thefirst translation value, the second translation value, and the firstrotation value. This problem may be avoided by collecting 2D scan dataat intermediate scan positions 1446. In an embodiment, the 2D scan datais collected and processed at regular intervals, for example, once persecond. In this way, features in the environment are identified insuccessive 2D scans at positions 1446. In an embodiment, when more thantwo 2D scans are obtained, the controller 1368 may use the informationfrom all the successive 2D scans in determining the translation androtation values in moving from the first position 1442 to the secondposition 1444. In another embodiment, only the first and last scans inthe final calculation, simply using the intermediate 2D scans to ensureproper correspondence of matching features. In most cases, accuracy ofmatching is improved by incorporating information from multiplesuccessive 2D scans.

It should be appreciated that as the scanner 1308 is moved beyond thesecond position 1444, a two-dimensional image or map of the environmentbeing scanned may be generated. It should further be appreciated that inaddition to generating a 2D map of the environment, the data fromscanner 1308 may be used to generate (and store) a 2D trajectory of thescanner 1308 as it is moved through the environment. In an embodiment,the 2D map and/or the 2D trajectory may be combined or fused with datafrom other sources in the registration of measured 3D coordinates. Itshould be appreciated that the 2D trajectory may represent a pathfollowed by the 2D scanner 1308.

Referring now to FIG. 17 , a method 1460 is shown for generating atwo-dimensional map with annotations. The method 1460 starts in block1462 where the facility or area is scanned to acquire scan data 1470.The scanning is performed by carrying the scanner 1308 through the areato be scanned. The scanner 1308 measures distances from the scanner 1308to an object, such as a wall for example, and also a pose of the scanner1308 in an embodiment the user interacts with the scanner 1308 viaactuator 1438. In the illustrated embodiments, the mobile device 1343provides a user interface that allows the operator to initiate thefunctions and control methods described herein. Using the registrationprocess desired herein, the two dimensional locations of the measuredpoints on the scanned objects (e.g. walls, doors, windows, cubicles,file cabinets etc.) may be determined. It is noted that the initial scandata may include artifacts, such as data that extends through a window1472 or an open door 1474 for example. Therefore, the scan data 1470 mayinclude additional information that is not desired in a 2D map or layoutof the scanned area.

The method 1460 then proceeds to block 1464 where a 2D map 1476 isgenerated of the scanned area. The generated 2D map 1476 represents ascan of the area, such as in the form of a floor plan without theartifacts of the initial scan data. It should be appreciated that the 2Dmap 1476 represents a dimensionally accurate representation of thescanned area that may be used to determine the position and pose of themobile scanning platform 1200 in the environment to allow theregistration of the 3D coordinate points measured by the 3D measurementdevice 1210. In the embodiment of FIG. 18 , the method 1460 thenproceeds to block 1466 where optional user-defined annotations are madeto the 2D maps 1476 to define an annotated 2D map that includesinformation, such as dimensions of features, the location of doors, therelative positions of objects (e.g. liquid oxygen tanks, entrances/exitsor egresses or other notable features such as but not limited to thelocation of automated sprinkler systems, knox or key boxes, or firedepartment connection points (“FDC”). In an embodiment, the annotationmay also be used to define scan locations where the mobile scanningplatform 1200 stops and uses the 3D scanner 1210 to perform a stationaryscan of the environment.

Once the annotations of the 2D annotated map are completed, the method1460 then proceeds to block 1468 where the 2D map is stored in memory,such as nonvolatile memory 1387 for example. The 2D map may also bestored in a network accessible storage device or server so that it maybe accessed by the desired personnel.

Referring now to FIGS. 19A-19C, an embodiment is shown of a laserscanner 1510. In this embodiment, the laser scanner 1210 has a measuringhead 1522 and a base 1524. The measuring head 1522 is mounted on thebase 1524 such that the laser scanner 1210 may be rotated about avertical axis 1523. In one embodiment, the measuring head 1522 includesa gimbal point 1527 that is a center of rotation about the vertical axis1523 and a horizontal axis 1525. The measuring head 1522 has a rotarymirror 1526, which may be rotated about the horizontal axis 1525. Therotation about the vertical axis may be about the center of the base1524. In one embodiment, the vertical axis 1523 is coaxial with thecenter axis of the post 1209. The terms vertical axis and horizontalaxis refer to the scanner in its normal upright operating position. Itis possible to operate a 3D coordinate measurement device on its side orupside down, and so to avoid confusion, the terms azimuth axis andzenith axis may be substituted for the terms vertical axis andhorizontal axis, respectively. The term pan axis or standing axis mayalso be used as an alternative to vertical axis.

The measuring head 1522 is further provided with an electromagneticradiation emitter, such as light emitter 1528, for example, that emitsan emitted light beam 1530. In one embodiment, the emitted light beam1530 is a coherent light beam such as a laser beam. The laser beam mayhave a wavelength range of approximately 300 to 1600 nanometers, forexample 790 nanometers, 905 nanometers, 1550 nm, or less than 400nanometers. It should be appreciated that other electromagneticradiation beams having greater or smaller wavelengths may also be used.The emitted light beam 1530 is amplitude or intensity modulated, forexample, with a sinusoidal waveform or with a rectangular waveform. Theemitted light beam 1530 is emitted by the light emitter 1528 onto a beamsteering unit, such as mirror 1526, where it is deflected to theenvironment. A reflected light beam 1532 is reflected from theenvironment by an object 1534. The reflected or scattered light isintercepted by the rotary mirror 1526 and directed into a light receiver1536. The directions of the emitted light beam 1530 and the reflectedlight beam 1532 result from the angular positions of the rotary mirror1526 and the measuring head 1522 about the axes 1525, 1523,respectively. These angular positions in turn depend on thecorresponding rotary drives or motors.

Coupled to the light emitter 1528 and the light receiver 1536 is acontroller 1538. The controller 1538 determines, for a multitude ofmeasuring points X, a corresponding number of distances d between thelaser scanner 1210 and the points X on object 1534. The distance to aparticular point X is determined based at least in part on the speed oflight in air through which electromagnetic radiation propagates from thedevice to the object point X. In one embodiment the phase shift ofmodulation in light emitted by the laser scanner 1210 and the point X isdetermined and evaluated to obtain a measured distance d.

The speed of light in air depends on the properties of the air such asthe air temperature, barometric pressure, relative humidity, andconcentration of carbon dioxide. Such air properties influence the indexof refraction n of the air. The speed of light in air is equal to thespeed of light in vacuum c divided by the index of refraction. In otherwords, c_(air)=c/n. A laser scanner of the type discussed herein isbased on the time-of-flight (TOF) of the light in the air (theround-trip time for the light to travel from the device to the objectand back to the device). Examples of TOF scanners include scanners thatmeasure round trip time using the time interval between emitted andreturning pulses (pulsed TOF scanners), scanners that modulate lightsinusoidally and measure phase shift of the returning light (phase-basedscanners), as well as many other types. A method of measuring distancebased on the time-of-flight of light depends on the speed of light inair and is therefore easily distinguished from methods of measuringdistance based on triangulation. Triangulation-based methods involveprojecting light from a light source along a particular direction andthen intercepting the light on a camera pixel along a particulardirection. By knowing the distance between the camera and the projectorand by matching a projected angle with a received angle, the method oftriangulation enables the distance to the object to be determined basedon one known length and two known angles of a triangle. The method oftriangulation, therefore, does not directly depend on the speed of lightin air.

In one mode of operation, the scanning of the volume around the 3Dmeasurement device 1210 takes place by rotating the rotary mirror 1526relatively quickly about axis 1525 while rotating the measuring head1522 relatively slowly about axis 1523, thereby moving the assembly in aspiral pattern. This is sometimes referred to as a compound mode ofoperation. In an exemplary embodiment, the rotary mirror rotates at amaximum speed of 5820 revolutions per minute. For such a scan, thegimbal point 1527 defines the origin of the local stationary referencesystem. The base 1524 rests in this local stationary reference system.In other embodiments, another mode of operation is provided wherein the3D measurement device 1210 rotates the rotary mirror 1526 about the axis1525 while the measuring head 1522 remains stationary. This is sometimesreferred to as a helical mode of operation.

In an embodiment, the acquisition of the 3D coordinate values furtherallows for the generation of a 3D trajectory, such as the 3D trajectory(e.g. 3D path) of the gimbal point 1527 for example. This 3D trajectorymay be stored and combined or fused with other data, such as data fromthe 2D scanner and/or from an inertial measurement unit for example, andused to register 3D coordinate data. It should be appreciated that the3D trajectory may be transformed from the gimbal point 1527 to any otherlocation on the system, such as the base unit.

In addition to measuring a distance d from the gimbal point 1527 to anobject point X, the laser scanner 1210 may also collect gray-scaleinformation related to the received optical power (equivalent to theterm “brightness.”) The gray-scale value may be determined at least inpart, for example, by integration of the bandpass-filtered and amplifiedsignal in the light receiver 636 over a measuring period attributed tothe object point X.

The measuring head 1522 may include a display device 1540 integratedinto the laser scanner 1210. The display device 1540 may include agraphical touch screen 1541, which allows the operator to set theparameters or initiate the operation of the laser scanner 1210. Forexample, the screen 1541 may have a user interface that allows theoperator to provide measurement instructions to the device, and thescreen may also display measurement results.

The laser scanner 1210 includes a carrying structure 1542 that providesa frame for the measuring head 1522 and a platform for attaching thecomponents of the laser scanner 1210. In one embodiment, the carryingstructure 1542 is made from a metal such as aluminum. The carryingstructure 1542 includes a traverse member 1544 having a pair of walls1546, 1548 on opposing ends. The walls 1546, 1548 are parallel to eachother and extend in a direction opposite the base 1524. Shells 1550,1552 are coupled to the walls 1546, 1548 and cover the components of thelaser scanner 1210. In the exemplary embodiment, the shells 1550, 1552are made from a plastic material, such as polycarbonate or polyethylenefor example. The shells 1550, 1552 cooperate with the walls 1546, 1548to form a housing for the laser scanner 1210.

On an end of the shells 1550, 1552 opposite the walls 1546, 1548 a pairof yokes 1554, 1556 are arranged to partially cover the respectiveshells 1550, 1552. In the exemplary embodiment, the yokes 1554, 1556 aremade from a suitably durable material, such as aluminum for example,that assists in protecting the shells 1550, 52 during transport andoperation. The yokes 1554, 1556 each includes a first arm portion 1558that is coupled, such as with a fastener for example, to the traverse1544 adjacent the base 1524. The arm portion 1558 for each yoke 1554,1556 extends from the traverse 1544 obliquely to an outer corner of therespective shell 1550, 1552. From the outer corner of the shell, theyokes 1554, 1556 extend along the side edge of the shell to an oppositeouter corner of the shell. Each yoke 1554, 1556 further includes asecond arm portion that extends obliquely to the walls 1546, 1548. Itshould be appreciated that the yokes 1554, 1556 may be coupled to thetraverse 1544, the walls 1546, 1548 and the shells 1550, 1554 atmultiple locations.

In an embodiment, on top of the traverse 1544, a prism 1560 is provided.The prism extends parallel to the walls 1546, 1548. In the exemplaryembodiment, the prism 1560 is integrally formed as part of the carryingstructure 1542. In other embodiments, the prism 1560 is a separatecomponent that is coupled to the traverse 1544. When the mirror 1526rotates, during each rotation the mirror 1526 directs the emitted lightbeam 1530 onto the traverse 1544 and the prism 1560. In someembodiments, due to non-linearities in the electronic components, forexample in the light receiver 1536, the measured distances d may dependon signal strength, which may be measured in optical power entering thescanner or optical power entering optical detectors within the lightreceiver 1536, for example. In an embodiment, a distance correction isstored in the scanner as a function (possibly a nonlinear function) ofdistance to a measured point and optical power (generally unscaledquantity of light power sometimes referred to as “brightness”) returnedfrom the measured point and sent to an optical detector in the lightreceiver 1536. Since the prism 1560 is at a known distance from thegimbal point 1527, the measured optical power level of light reflectedby the prism 1560 may be used to correct distance measurements for othermeasured points, thereby allowing for compensation to correct for theeffects of environmental variables such as temperature. In the exemplaryembodiment, the resulting correction of distance is performed by thecontroller 1538.

In an embodiment, the base 1524 is coupled to a swivel assembly (notshown) such as that described in commonly owned U.S. Pat. No. 8,705,012('012), which is incorporated by reference herein. The swivel assemblyis housed within the carrying structure 1542 and includes a motor thatis configured to rotate the measuring head 1522 about the axis 1523. Inan embodiment, the angular/rotational position of the measuring head 622about the axis 1523 is measured by angular encoder. In the embodimentsdisclosed herein, the base (with or without the swivel assembly) may bemounted to the post 1209.

An auxiliary image acquisition device 1566 may be a device that capturesand measures a parameter associated with the scanned area or the scannedobject and provides a signal representing the measured quantities overan image acquisition area. The auxiliary image acquisition device 1566may be, but is not limited to, a pyrometer, a thermal imager, anionizing radiation detector, or a millimeter-wave detector. In anembodiment, the auxiliary image acquisition device 1566 is a colorcamera.

In an embodiment, a central color camera (first image acquisitiondevice) 1512 is located internally to the scanner and may have the sameoptical axis as the 3D scanner device. In this embodiment, the firstimage acquisition device 1512 is integrated into the measuring head 1522and arranged to acquire images along the same optical pathway as emittedlight beam 1530 and reflected light beam 1532. In this embodiment, thelight from the light emitter 1528 reflects off a fixed mirror 1516 andtravels to dichroic beam-splitter 518 that reflects the light 1517 fromthe light emitter 1528 onto the rotary mirror 1526. In an embodiment,the mirror 1526 is rotated by a motor 1537 and the angular/rotationalposition of the mirror is measured by angular encoder 1534. The dichroicbeam-splitter 1518 allows light to pass through at wavelengths differentthan the wavelength of light 1517. For example, the light emitter 1528may be a near infrared laser light (for example, light at wavelengths of780 nm or 1150 nm), with the dichroic beam-splitter 1518 configured toreflect the infrared laser light while allowing visible light (e.g.,wavelengths of 400 to 700 nm) to transmit through. In other embodiments,the determination of whether the light passes through the beam-splitter1518 or is reflected depends on the polarization of the light. Thedigital camera 1512 obtains 2D images of the scanned area to capturecolor data to add to the scanned image. In the case of a built-in colorcamera having an optical axis coincident with that of the 3D scanningdevice, the direction of the camera view may be easily obtained bysimply adjusting the steering mechanisms of the scanner—for example, byadjusting the azimuth angle about the axis 1523 and by steering themirror 1526 about the axis 1525. One or both of the color cameras 1512,1566 may be used to colorize the acquired 3D coordinates (e.g. the pointcloud).

In an embodiment, when the 3D scanner is operated in compound mode, acompound compensation may be performed to optimize the registration ofdate by combining or fusing sensor data (e.g. 2D scanner, 3D scannerand/or IMU data) using the position and orientation (e.g. trajectory) ofeach sensor.

It should be appreciated that while embodiments herein refer to the 3Dscanner 1210 as being a time-of-flight (phase shift or pulsed) scanner,this is for exemplary purposes and the claims should not be so limited.In other embodiments, other types of 3D scanners may be used, such asbut not limited to structured light scanners, area scanners,triangulation scanners, photogrammetry scanners, or a combination of theforegoing.

Referring now to FIGS. 20-21 , an embodiment is shown of a method 1600for scanning an environment with the mobile scanning platform 1200. Themethod 1600 starts in block 1602 where the platform is configured. Inthe embodiment where the platform is platform 1200, the configuring mayinclude attaching the 2D scanner 1308 to the respective arm or holder,and the 3D measurement device 1210 to the post 1209. In an embodimentwhere the platform is semi-autonomous or fully autonomous, theconfiguring may include determining a path for the platform 1200 tofollow and defining stationary scan locations (if desired). In anembodiment, the path may be determined using the system and methoddescribed in commonly owned U.S. patent application Ser. No. 16/154,240,the contents of which are incorporated by reference herein. Once thepath is defined, the 2D scanner 1308 and 3D scanner 1210 may be coupledto the platform 1200. It should be appreciated that in some embodiments,the platform 1200 may be remotely controlled by an operator and the stepof defining a path may not be performed.

Once the platform 1200 is configured, the method 1600 proceeds to block1604 where the 2D scanner 1308 is initiated and the 3D measurementdevice 1210 is initiated in block 1606. It should be appreciated thatwhen operation of the 2D scanner 1308 is initiated, the 2D scannerstarts to generate a 2D map of the environment as described herein.Similarly, when operation of the 3D measurement device 1210 isinitiated, the coordinates of 3D points in the environment are acquiredin a volume about the 3D scanner.

The method 1600 then proceeds to block 1608 where the platform 1200 ismoved through the environment. As the platform 1200 is moved, both the2D scanner 1308 and the 3D measurement device 1210 continue to operate.This results in the generation of both a 2D map 1610 (FIG. 21 ) and theacquisition of 3D points 1611. In an embodiment, as the 2D map isgenerated, the location or path 1612 of the platform 1200 is indicatedon the 2D map. In an embodiment, the platform 1200 may include a userinterface that provides feedback to the operator during the performingof the scan. In an embodiment, a quality attribute (e.g. scan density)of the scanning process may be determined during the scan. When thequality attribute crosses a threshold (e.g. scan density too low), theuser interface may provide feedback to the operator. In an embodiment,the feedback is for the operator to perform a stationary scan with the3D scanner.

The method 1600 then proceeds to block 1614 where the acquired 3Dcoordinate points are registered into a common frame of reference. Itshould be appreciated that since the platform 1200 is moving while the3D measurement device 1210 is acquiring data, the local frame ofreference of the 3D scanner is also changing. Using the position andpose data from the 2D scanner 1308, the frame of reference of theacquired 3D coordinate points may be registered into a global frame ofreference. In an embodiment, the registration is performed as theplatform 1200 is moved through the environment. In another embodiment,the registration is done when the scanning of the environment iscompleted.

The registration of the 3D coordinate points allows the generation of apoint cloud (e.g. a collection of 3D coordinates or points in 3D space)in block 1618. In an embodiment, a representation of the path 1620 ofthe platform 1200 is shown in the point cloud 1616. In some embodiments,the point cloud 1616 is generated and displayed to the user as theplatform 1200 moves through the environment being scanned. In theseembodiments, blocks 1608, 1614, 1618 may loop continuously until thescanning is completed. With the scan complete, the method 1600 ends inblock 1622 where the point cloud 1616 and 2D map 1610 are stored inmemory of a controller or processor system

The term “about” is intended to include the degree of error associatedwith measurement of the particular quantity based upon the equipmentavailable at the time of filing the application.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, element components,and/or groups thereof.

While the disclosure is provided in detail in connection with only alimited number of embodiments, it should be readily understood that thedisclosure is not limited to such disclosed embodiments. Rather, thedisclosure can be modified to incorporate any number of variations,alterations, substitutions or equivalent arrangements not heretoforedescribed, but which are commensurate with the spirit and scope of thedisclosure. Additionally, while various embodiments of the disclosurehave been described, it is to be understood that the exemplaryembodiment(s) may include only some of the described exemplary aspects.Accordingly, the disclosure is not to be seen as limited by theforegoing description, but is only limited by the scope of the appendedclaims.

What is claimed is:
 1. A system comprising: a coordinate scannerconfigured to optically measure and determine a plurality ofthree-dimensional coordinates to a plurality of locations on at leastone surface in the environment, the coordinate scanner being sized to becarried by a single person and moved through the environment whileacquiring the plurality of three-dimensional coordinates; a displayremovably coupled to the coordinate scanner, the display having agraphical user interface; one or more processors configured to:determine a quality attribute of a process of measuring the plurality ofthree-dimensional coordinates based at least in part on the movement ofthe coordinate scanner in the environment; and display a graphicalquality indicator on the graphical user interface based at least in parton the quality attribute, the quality indicator is a graphical elementhaving at least one movable element; wherein the graphical elementfurther comprises a quality symbol, the at least one changeable barindicating a first quality attribute, the quality symbol indicating asecond quality attribute; wherein the first quality attribute is basedat least in part on the speed of the coordinate scanner, and the secondquality attribute is based at least in part on a target attribute; andwherein the target attribute is the age of the target, the age of thetarget being being the number of successive images a natural feature orartificial target appear in a plurality of successive images acquired bythe coordinate scanner.
 2. The system of claim 1, wherein the size ofthe movable element is based on the quality attribute.
 3. The system ofclaim 1, wherein the at least one movable element includes a pluralityof stacked bars in the movable element.
 4. (canceled)
 5. (canceled) 6.The system of claim 1, wherein the graphical element further comprises aquality symbol, the one or more processors being further configured tochange the quality symbol based on the quality attribute.
 7. The systemof claim 1, wherein the parameter is one of a length of the at least onemovable element, a color of the at least one movable element, or acombination thereof.
 8. The system of claim 1, wherein: the graphicalelement is a first color when the process of measuring the plurality ofthree-dimensional coordinates with the coordinate scanner provides adensity of the plurality of three-dimensional coordinates above a firstthreshold; the graphical element is a second color when the process ofmeasuring the plurality of three-dimensional coordinates with thecoordinate scanner provides the density of the plurality ofthree-dimensional coordinates below a second threshold; and thegraphical element is a third color when the process of measuring theplurality of three-dimensional coordinates with the coordinate scannerprovides the density of the plurality of three-dimensional coordinatesbeing between the first threshold and the second threshold.
 9. Thesystem of claim 1, wherein the quality attribute is based at least inpart on a translational speed of the coordinate scanner through theenvironment.
 10. The system of claim 1, wherein the quality attribute isbased at least in part on at least one of: a number of tracking targets;an age of the targets; a rotational speed of the coordinate scanner; aquality threshold of images used to track the coordinate scanner; anumber of three-dimensional points acquired; a 3D geometry of theenvironment; a distance to the objects being scanned; and a level ofnoise in the plurality of three-dimensional coordinates.
 11. The systemof claim 1, wherein the graphical user interface includes a firstportion and a second portion, the quality indicator being positioned inthe first portion.
 12. The system of claim 1, wherein the qualityindicator instructs the operator to perform a stationary scan.
 13. Thesystem of claim 1, wherein the quality indicator instructs the operatorto perform an anchor scan.
 14. The system of claim 1, wherein thequality indicator instructs the operator to record an anchor object. 15.The system of claim 1, wherein the one or more processors are furtherconfigured to determine when tracking has been lost, and displaying onthe graphical user interface two overlapping transparent images of theenvironment.